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Abstract 

A token ring is an arrangement of n processors that take turns engaging in an activity 
which must be controlled. A token confers the right to engage in the controlled activity. 
Processors communicate with neighbors in the ring to obtain and release a token. The 
communication mechanism investigated in this paper is the safe register abstraction, 
which may arbitrarily corrupt a value that a processor reads when the operation reading a 
register is concurrent with an write operation on that register by a neighboring processor. 
The main results are simple protocols for quasi-atomic communication, constructed from 
safe registers. A quasi-atomic register behaves atomically except that a special ± value 
may be returned in the case of concurrent read and write operations. Under certain 
conditions that constrain the number of writes and registers, quasi-atomic protocols are 
adequate substitutes for atomic protocols. The paper demonstrates how quasi-atomic 
protocols can be used to implement a self-stabilizing token ring, either by using two 
safe registers between neighboring processors or by using Oilgn) safe registers between 
neighbors, which lowers read complexity. 
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The fundamental task in computing is to implement 
higher-level operations with lower-level ones. 

— Leslie Lamport 



1 Introduction 

Among the many qualitative dimensions characterizing distributed computing are time and 
communication types. Time, whether in processing rate, duration, or delay of communication 
operations, may be modeled synchronously or asynchronously; types of communication in- 
clude transient (message passing), persistent (shared memory) [6], and rendezvous [3]. Asyn- 
chronous models pose the most challenging problems for reasoning about program properties. 
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particularly when failures are considered. The model of shared objects with prescribed op- 
erations captures the essence of persistent communication between asynchronous processes. 
The most primitive type of shared object is a register with only two operations, read and 
write. 

Emerging large scale platforms, notably multicore architectures and cloud computing fa- 
cilities, motivate relaxed consistency operations, nonblocking semantics, and speculative or 
probabilistic approaches. Register abstractions are of potential interest for two reasons: first, 
registers are wait-free operations which are meaningful at both low-level (machine architec- 
ture) and high-level design (manipulating key- value pairs); second, the literature on registers 
has explored numerous models of concurrency restriction and degraded semantics, finding 
constructions that overcome deficiencies of unreliable read operations. 

Register properties can be axiomatized [HI O [6], with several choices for behavior dur- 
ing concurrent operations; different choices lead to stronger or weaker register types. The 
strongest type is an atomic register, and the weakest type is a safe register. Atomic registers 
are most useful for applications, because they simplify reasoning in the face of concurrent 
execution. Safe registers are most convenient for implementors, because they have minimal 
requirements on behavior under concurrent execution. A significant literature of protocols 
and constructions explores how atomic behavior can be derived from safe registers or other 
shared objects with weak semantics. Such constructions are typically complex, from a re- 
source standpoint (many low-level registers needed to implement a higher-level atomic one) 
or from a verification standpoint. 

Contributions. Protocols presented in this paper show how a self-stabilizing token ring 
can be implemented using safe registers, which are the weakest type in Lamport's register hi- 
erarchy [5l [6] . Previous work showed that regular and safe registers suffice for communication 
in a self-stabilizing token ring |I4j; the contributions of the new protocols are an improved 
validation framework and a construction that uses two safe registers rather than 0{\gn) 
safe registers between neighbors. The safe register protocols for read and write operations 
are simple, thanks to the closed-loop nature of the stabilizing token ring, which inherently 
limits concurrency. One may question whether exploiting a concurrency-limiting property is 
interesting, since the point of wait-free operations is to allow unrestricted concurrency. In 
fact, many high-level tasks have some sequential or concurrency-limiting properties, and it 
is sensible to exploit such properties if they simplify lower-level design. Moreover, retaining 
wait-free behavior of low-level operations can benefit implementation designs (which might 
use speculation, caching, and other ideas) even when higher-level tasks are sequential. 
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Organization. Section [2] briefly reviews terminology for register abstractions and the token 
ring, and Section [3] casts self-stabilizing token passing in terms of atomic registers. Then 
Section H] introduces quasi-atomicity and a protocol implementing quasi-atomic operations 
using safe registers (subsection I4.ip . Two following sections, [5] and [6l present self-stabilizing 
adaptations of the token ring using the quasi-atomic constructions. Discussion wraps up the 
paper in Section [71 

2 Preliminaries 

This section informally reviews terminology of registers, constructions from registers, and a 
self-stabilizing token ring protocol. An atomic register is a shared object with two methods, 
read and write. The time between invocation of a register method and its response can be 
arbitrary in duration, which allows for interleaving of steps from different processors in an 
execution. Two operations are considered to be concurrent if the invocation of one occurs in 
the interval between invocation and response of the other. Formal verification of protocols 
consists of mapping an interleaved execution to a linearized history of processor steps and 
register operations in such that each invocation of a register method is immediately followed 
by its response in the history; the verification arguments in this paper are informal, reasoning 
at the level of operation properties rather than constructing linearized mappings. Additional 
nomenclature is given in Section H] for reasoning about operation intervals. In executions 
without concurrent register operations, behavior of read is unambiguous: the response to any 
read is the value most recently written to that register. 

To describe atomic behavior operationally, consider a write invocation W(x) on a register 
R which contains the value y prior to \N{x), where x ^ y. Value y is called the old value, 
and X is the new value. A register is atomic if any read not concurrent with a write responds 
with the most recently written value, and read operations concurrent with a write responds 
with either y or x, subject to the constraint that once a read returns x, any subsequent read 
also returns x. 

A regular register weakens atomicity somewhat: a read concurrent with a write may return 
the old or new value arbitrarily. A safe register weakens atomicity further, only guaranteeing 
that a read concurrent with a write returns some value in the domain of the register (binary, 
m-bit integer, or whatever the capacity is given for the register). It is perhaps surprising that 
safe registers could be useful, until one sees that the definitions of regular and safe collapse 
for the case of a single-bit register, provided that is only written when the current value needs 
to be changed. One way to specify an atomic register is add a constraint to a regular register: 
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an atomic register is a regular register without new-old inversion, that is, the old value is not 
returned once the new value has been returned in a sequence of read operations concurrent 
with a write. 

Register properties become more complex when many processors read and write the same 
register. The notation mWnR indicates that m processors, called writers, and n processors, 
called readers, may concurrently have operations on the same register. For the token ring pro- 
tocols in this paper, communication is confined to a ring in which each processor only shares 
registers with neighbors in the ring; moreover, communication is unidirectional, because a 
token is consistently passed from each processor to only one other. The type of register could 
be IWIR, except that self-stabilization invalidates the assumption that a writer's internal 
state correctly estimates the value of the register to be written — such an assumption is im- 
portant to avoid writing except when needed to change a value. Therefore, the protocols use 
1W2R registers, so that the processor writing can also read that register. 

3 Dij using Registers 

The vehicle for demonstrating quasi-atomic registers is a self-stabilizing token ring protocol 
[H [2]. This protocol is a simple construction that depends on atomic communication for 
the self-stabilization property. The protocol was originally expressed as a ring of processes 
communicating through shared state variables; subsequent work adapted the protocol to a 
register model of communication [isj^ . A register-based adaptation of this famous protocol 
is shown in Figure [TJ We call this the Dij protocol in the remainder of the paper. 



1 Dij,(i^): 
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local variables x, y 
do forever 
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if^T^O A X ^ y then 
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7 



critical section 
ifi = A y = X then 
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10 



critical section 
write X to output register R 



Figure 1: register-based Dij protocol for pi 



^Lamport also introduced an adaptation of Dij to a common shared memory model in [8]. 
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Figure [T] describes the behavior of processor pQ (hnes 8-10) and the behavior of processors 
pi-pn-i (hnes 5-7). The ring uses unidirectional communication, as each processor reads a 
register (hne 4) written by the previous processor in the ring (writing occurs on hne 11). 
The token abstraction is embodied by conditions on hnes 5 and 8, which ahow a processor to 
execute a "critical section" representing some activity to be controlled, like mutual exclusion. 
Terms piQi and p^ei denote the previous and next processors, with respect to pi, in the 
ring. The registers are supposed to be atomic. Variables x, y, and the registers may have 
arbitrary initial values, however the domain of all variables and registers is confined to the 
set {i \ < i < K}, where K is some given constant satisfying K > 2n. The proof of 
self-stabilization for Dij is typically shown by defining a subset of the state-space of the ring 
of processors called the legitimate set, showing that this set is closed under execution (each 
successor of a legitimate state is legitimate) , and that it satisfies safety and liveness properties 
(the token perpetually advances in the ring, and there is always a single token). Convergence 
from an arbitrary initial state consists of showing (by contradiction) the absence of deadlock, 
e.g. that po must infinitely often execute line 9, and that eventually the assignment on line 
9 obtains a value different from that in any other variable or register throughout the ring. 

We suppose in the sequel that the reader is familiar with stabilization arguments \12\ [T3] 
for Dij, and confine our task to replacing the atomic registers used in Figured] by constructions 
using safe registers. This paper does not attempt to settle fundamental questions about 
possibility or impossibility of token circulation using safe registers; for instance, we do not 
explore the space of algorithms that communicate bidirectionally between neighbors in the 
ring, nor do we investigate probabilistic register constructions. Rather, we take the Dij 
protocol as the given structure to implement, and consider how it can be adapted to safe 
register communication. 

Transforming Dij from its original shared state model to using atomic link registers is 
straightforward, and using regular registers instead of atomic ones isn't a challenging problem. 
Safe registers, however, require more interesting protocols because these registers have weak 
concurrency properties. The only guarantee by a safe register is that a read not concurrent 
with a write will return the most recently written value. A read concurrent with a write can 
return any value in the register's domain, even if the value being written is already equal to 
what the register contains. Two of the difficulties in constructing a transformation are neatly 
summarized in the following conjectures. 

Conjecture 3.1 Algorithm Dij cannot be implemented using only one safe register between 
Pi andpicQi. 

The intuition for this conjecture is that a processor with only one safe register must write 
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to that register in some case, and the reader can have unboundedly many reads concurrent 
with such a write operation, each resulting in an arbitrary value. 

Conjecture 3.2 Algorithm Dij cannot be implemented using only IWIR safe registers be- 
tween Pi and Pi(Qi ■ 

The intuition for this second conjecture is that a processor cannot ascertain the value of its 
output register, and therefore must continually rewrite it, but doing so admits the possibility 
of having every register read being concurrent with a write, which would defy progress. 

If conjecture [3TT] holds, then any transformation will have pi write more than one register 
that pj0i reads. Section [5] provides a transformation using two registers for each processor in 
the ring, which would be optimal if the conjecture holds. If conjecture 13.21 holds, then any 
transformation will allow that the writers of registers can also read the values of the registers 
they write: these safe registers are 1W2R registers. Since no processor can read and write 
the same register concurrently, any read by pi of its own output register is trivially atomic. 

4 Quasi- Atomic Registers 

To the standard terminology mentioned in previous sections, a variation of the atomicity 
property is used in protocols of later sections. Quasi-atomic behavior differs from atomic 
behavior only in that a read operation may return the exception value _L, indicating a "busy" 
condition where the reader should retry the operation. Similarly, let a quasi-regular register 
differ only from a regular register by allowing a read to respond with _L. In the absence of 
concurrency, the special _L value is not returned by a read. 

Reasoning about nonatomic register operations is often explained with diagrams and 
ordering relations. Diagrams illustrate how register operations have duration, and how the 
time intervals of the operations are related. Figure [2] shows a typical case of two consecutive 
write operations, W and W ^ both due to some processor p writing to the same register, and 
two consecutive read operations, R and ii', of that register by another processor p' . The 
figure shows that W ends before R' begins; thus W precedes R', written W -< R'. We write 
W ^ R if W starts before R starts: either W ~< R or the two operations are concurrent. In 
the case of Figure [21 W ^ R and R ^ W'. Protocols for high-level operations usually include 
numerous register operations by each processor. For instance, the two write operations of 
processor p in Figure [2] could be due to some higher-level procedure call, which has a duration 
spanning the intervals of W and W . The interval from the start of W to the end of W is 
said to contain the interval of R, because W begins before R and W ends after R ends. 
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Figure 2: write and read operations may overlap 

Some simple algebra on the relations between register operation intervals aids in reasoning 
about register protocols. The -< relation is transitive and containment is also transitive. 
Concurrency is not transitive: A being concurrent with B and B being concurrent with C 
does not imply that A and C are concurrent. The ^ relation is not transitive, however some 
combinations are transitive, for instance A ^ B A B ~< C A < C holds because A 
begins before B ends, hence A begins before C begins, which implies A < C. The following 
inference about containment is used later in this section. 

Observation 4.1 Suppose A ■< Bq, Bj -< iJj+i for < j < m, and Bm ^ A' . Then the 
time interval from the start of A to the end of A' contains an interval that begins with some 
time instant in Bq and ends with some point in Bm- 

4.1 Duplicate Writes and A;-Scan Reads 

We propose here a protocol that, under certain conditions, transforms safe registers to quasi- 
atomic behavior for communication from pi to The proposed protocol consists of an 
AWrite procedure invoked by pi and an ARead(A;) procedure invoked by Pi©i. Figure [3] shows 
the two procedures, which use a pair of 1W2R registers between pi and An AWrite(wa/) 
invocation writes val to registers and Rf,, but only if these registers do not already both 
contain val. An ARead(/c) invocation reads both of these registers k times in succession, 
returning _L if not all of the read operations yield the same value, and otherwise returning 
the (unanimous) value from the registers. On one hand, value _L indicates a reading failure, 
that is, _L is returned when it is known that ARead(/c) could not return a value with atomic 
read semantics. On the other hand, when ARead(A;) does not return _L we cannot be sure 
that the returned value is an atomic read of the latest value from an AWrite operation. The 
following lemma finds a condition for which ARead(A;) is quasi-atomic. 

Lemma 4.1 In any execution where pi invokes AWrite at most {k — 1) times, then every 
ARead(k) invocation is a quasi-atomic read by pi^^i. 

Proof: The proof begins by showing that any ARead(/c) is quasi-regular, that is, it either 
returns the value of the registers prior to any AWrite commencing, or the value of the registers 
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1 AWrite(m/): 

2 local variables A, B 

3 read from into A 

4 read from Rf, into B 

5 \i A = B = val then return 

6 else 

7 write val to R^ 

8 write val to Rft 

9 return 

10 ARead(A;): 

11 local array A[k]^ B[k] 

12 for i = \ to k: 

13 read from Rq into A[i] 

14 read from R;, into B[i\ 

15 if all of A[.] and B[.] have same value 

16 then return 

17 else return _L 



Figure 3: duplicate write, fc-scan read protocol 

after some AWrite is finished and before the next AWrite starts, or the value _L. Then this 
argument is generalized to show that in any sequence of ARead(/c) invocations, no new- 
old inversion occurs. We show that any ARead(A;) is regular by contradiction, after first 
introducing a graph to represent the interaction between AWrite and ARead(fc) operations on 
the safe registers. 

To disambiguate AWrite invocations that may have the same val argument (see Figure 
ED, we assume that each AWrite is invoked to write a value distinct from all other (at most 
k — 2) AWrite invocations. Giving each AWrite a different input value from the previous AWrite 
presents a worst case execution with regard to the number of low- level writes. At the end of 
the proof, we examine cases where this assumption does not hold. 

Consider a single write operation to a safe register and a possibly concurrent read opera- 
tion on that register. Three possibilities are (i) the read returns the old value of the register 
(that is, the value that the register holds prior to the write), [ii) the new value of the register 
(that is, the register's value after the write is complete), or {Hi) an arbitrary value returned 
because the read operation is concurrent with the write operation. A sequence of {k — 1) 
AWrite invocations produces a sequence of writes to Rq and R;, registers, which we denote as 

wl wl ■■■ wt^ (1) 
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For cases the values of the register are of concern. Instead of looking at the sequence 

of write operations, we therefore examine the sequence 




(2) 



which distinguishes all possible situations that read operations on registers Ra and R;, may 
encounter during an execution. Term v} represents the situation where no writing has begun. 
Term w\ represents the interval of W^, which can yield an ambiguous value; signifies that 



the situation where and have finished, but W"^ has not yet started. Any ARead(fc) 
operation induces a sequence of read operations on and Rfe, 



The sequence of read operations ^ is related to sequence ([2]). A convenient portrayal of this 
relation is the following graph. First, let the terms of ([3]) be one set of vertices, and the terms 
of ^ are another set of vertices. The relation is given by adding edges between these two 
sets to form a bipartite graph induced by values returned from read operations. For example, 
if i?^ is concurrent with a write operation in the execution and returns a value different from 
Ra's initial content and different from any val previously written to Rq, then there is an edge 
between and some vertex. If instead reads the value between w^'s completion and 
vo^ starting, there is an edge between R'^ and one of {v^ ^w\,v?}. We say that an i?- vertex 
maps to a or u vertex according to the constructed graph. In addition to edges between 
vertices of ([3]) and ([2]), let edges also be added to the graph between successive items in each 
respective sequence: {v}-,w\), {w\,v^), ... , are edges; and {R\,R\), {R\,R^), . . . , are edges. 
The resulting graph is planar: the edges mapping i?-vertices to vertices from ([2]) do not cross 
(cases (a)-(d) below explain this point). 

With aid of the bipartite graph between reads and writer situations, we return the proof 
of the lemma, which is an implication, proved here by contradiction. A refutation of the 
lemma supposes an ARead(A;) returns an arbitrary non-_L value, that is, a value that does 
not correspond to any of {n* | 1 < i < A;}; terms {u* [ 1 < i < A;} represent intermediate 
points where Ra ^ R;,, and ARead(A;) would return _L, giving a contradiction. It follows 
that every safe-register read operation returns the same arbitrary value x, different from 
the value corresponding to any of {u* | 1 < i < k}. Therefore each term of the form i?^ 
maps to a vertex in { | 1 < i < A;}, and each term of the form i?^ maps to a vertex in 
{w\, I \ <i < k}. Sets { I 1 < i < A;} and { i?^ | 1 < j < A;} each have k vertices, whereas 
\{w\i \ ^ ^ i < k}\ = k — 1 and | 1 < i < A;}| = A; — 1. Some elementary observations 



is finished, but has not started. Term wl represents the interval of W^. Term is 



r?l J?2 p2 ryk r>k 

jTLf^ ■ ■ ■ it^ 



(3) 
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about the ordering of read and write operations constrain mapping from read operations to 
([2]) vertices, as follows. 

(a) i?^ and map to distinct vertices because the former maps to a write of Rq and the 
latter to a write of R;,. 

(b) For and Rl, an edge from R^ to implies that the edge from Rj^ to satisfies 
n > m by the ordering of the sequence of write operations. 

(c) For R{^ and R^^, i > j, with R{^ mapping to and i?^ mapping to w2, the sequential 
ordering of the read operations implies n > m {a similar observation holds for i?^ 
operations). 

(d) Observation (c) can be strengthened to n > m, because between any two read operations 
on Ra there is a read operation on Rf,, and observations (a) and (b) constrain the 
mapping targets to be distinct. 

By induction, for any h > j, R^ maps to a vertex distinct from the vertices that i?^ and 
R-'^ map to. Since the number of R vertices is 2k and the number of w vertices is 2[k — 1), 
the distinctness constraint mapping R vertices to w vertices implies a contradiction. This 
contradiction shows that any ARead(A;) returns a value that is either the initial value of the 
R-registers or a value that was written by some AWrite operation preceding the ARead(A;) 
or concurrent with the ARead(fc) operation. If the value is due to an AWrite preceding the 
ARead(^), then it must be the last such AWrite, because safe registers return the most recently 
written value in the absence of concurrency. Therefore, the protocol is quasi-regular. 

Proof of quasi-atomicity consists of showing that ordered ARead(fc) invocations do not 
exhibit new-old inversion. Suppose that the sequence of arguments to the {k — 1) AWrite 
operations ^ (let be the initial value of Rq and R;,). Consider two 

ARead(A) invocations A, A' , such that A' occurs after A, both with non-_L responses, and A' 
returns while A returns . New-old inversion occurs if j < i. However, j <i contradicts 
planarity of the graph. 

The arguments above verify the proof obligation when each AWrite has a distinct value; we 
now consider executions where AWrite invocations get repeated values. The simplest scenario 
is when consecutive AWrite invocations have the same value: in such cases, repeated AWrite 
invocations are not effective, because line 5 of Figure [3] is an early exit. Thus we focus on 
executions where repeated AWrite values are not consecutive. Here, there can be ambiguity 
in mapping low-level i?- vertices to register situations. However, the behavior of ARead(A;) in 
Figure [3] does not depend on values read (other than returning _L when values differ) , thus 
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the mapping under the assumption of uniquely written values remains valid. Note that an 
ARead(A;) concurrent with several AWrite operations, say W^, W^, W^, could read from low- 
level writes of and low-level writes of W^, where both and are effective and write 
the same value v. In such a case, it is possible that the ARead(/i;) returns v, picking up some 
instances of v from and some from W^. This does not violate quasi-regular behavior, 
since the value returned would be due to a concurrent AWrite. The planarity argument for 
successive ARead(/c) operations verifies quasi-atomic behavior. □ 

Corollary 4.2 In any execution where each ARead(k) by Pis)i is concurrent with at most 
{k — 1) AWrite operations of pi, all of pi^i 's ARead(k)s are quasi-atomic. 

Proof: Any finite execution has some number of pj's AWrite operations, and Lemma l4.1f s 
graph representation of low-level register situations and read operations applies to this ex- 
ecution. Each ARead(A;) operation comprises a sequence of low-level reads, which induces a 
subgraph for which the conditions of Lemma [4.11 hold . Thus, each ARead(A;) has quasi-atomic 
behavior. □ 

With respect to a single AWrite, an ARead(l) could return a non-_L value that is neither 
the old (the values of Rq and Rf, before the AWrite) nor the new value; instead, the ARead(l) 
returns an invalid value that we call contaminated. The number of contaminated ARead(l) 
operations following an AWrite is limited, and this fact can be exploited in protocols. The 
following lemma characterizes contamination. 

Lemma 4.3 In any execution where pi invokes AWrite at most m ■ k times, the number of 
contaminated ARead(k) operations is at most m. 

Proof: Consider the planar graph construction of Lemma 14.11 representing register situa- 
tions, applied to the execution from the (at most) m ■ k AWrite operations by pi and some 
number t > mof ARead(A;) operations invoked by Looking to find contradiction, suppose 
s of the ARead(A;) operations are contaminated, m < s < t. For a contaminated ARead(A;), 
in the graph all the read operations map to corresponding write operations representing read 
concurrent with write, so that these read operations return invalid results. This implies that 
the ARead(A;)'s low-level read operations map to k distinct vertices, because each of the k 
iterations (line 12, Figure [3|) scans both Rq and Rf,, and each of these is presumed concurrent 
with a write to that register. The lemma follows because no two ARead(A;) invocations have 
operations mapping to a common vertex: the first ARead(^) operation is an Ra mapping to 
a Wa, the last A Read (A;) operation maps to a Wb, and planarity excludes mapping to common 
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vertices between the first and last of tliese read operations. From m ■ k AWrites, there are 
2m • k low-level vertices, and with s > m ARead(A;) operations, there are s ■ 2k i?- vertices, 
thus s > m contradicts distinct mapping from i?- vertices to t(;-vertices. □ 



5 Two-Register Adaptation of Dij 

The quasi-atomic register protocol of Section [^J] supports transformation of the Dij protocol 
to the safe register model. In the transformed protocol, there are registers Rq and between 
each consecutive pair pi, in the ring. We call the registers that pi writes the output 

registers. Figure H] presents the two-register protocol for processor pi, < i < n. In this 
protocol, processors do not have durable states: in each cycle of the loop (lines 4-12), processor 
Pi reads output register Rq into a local variable (line 4). The output registers are written by 
the AWrite invocation at the end of the cycle (line 12). The reading by of pj's output 
registers occurs when invokes ARead(A;) (line 5). 



Two constants need to be set for the protocol, (p and K. It is sufficient that K > 2n, using 
standard verification arguments about Dij. Below, we derive a constraint for <j) to ensure that 
AWrite and ARead((/>) invocations behave quasi-atomically (a safety property), and later show 
that any sequence of ARead(0) invocations returning _L is bounded (a progress property). For 
the following lemma, an AWrite(rE) invocation is called effective in case x differs from the value 
of the output registers; an ineffective write merely reads the output registers, finding they 
already contain x, and returns. 




12 AWrite(x) 



Figure 4: two register D\]{(j),K) protocol for processor pi 
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Lemma 5.1 In any execution of the protocol of Figure with 4> > 2n, no invocation of 
ARead((j)) has a contaminated response. 



Proof: The lemma is shown by contradiction, assuming that in some execution E there is an 
ARead(0) by processor pi returning a contaminated value. The contradiction is demonstrated 
by deducing that the contaminated ARead((/>) at pi is concurrent with an AWrite invocation, 
also at Pi (which is impossible because no processor concurrently invokes both ARead and 
AWrite). 

To set up the contradiction, we consider the first contaminated ARead ((/>) in occur- 
ring at processor pj, and apply Lemma 14.11 to infer that processor piQi invoked at least (p 
effective AWrites, so that each of the ARead (i;^)'s register operations was concurrent with a 
corresponding write by PiQi- Figure [5] depicts the situation, where the vertical dotted lines 
indicate concurrent read and write operations; for instance, r\ and w\ are concurrent. 



Pi 



\ / 



Figure 5: situation for pi and PiQ\ 

The figure labels Piei's write operations wl, and so on, however it may be that 
and wl do not belong to the same AWrite. The figure is thus unlike the labeling of ([2]), 
because the labeling wl^, wl, w"^, . . . , wf comprise a subsequence of low- level register writes 
selected for the counterexample, to be concurrent with read operations. There could, in fact, 
be numerous effective AWrite operations between wl and w^^^. The figure also shows some 
ARead invocations by PiQi, labeled as TZ^, TZ'^~^. This follows from the logic of the 
protocol in Figured! in which any AWrite at line 12 is followed by an ARead on line 5. The 
arrows between w and TZ items in the figure signify precedence: wl -< TZ^, for example. The 
dashed arrow from to wl represents ^ wl, which holds because must end before 
wl ends so that can be concurrent with wl- Just as there could be numerous AWrites 
between successive u)-vertices in the figure, there could be other ARead invocations by piQi 
not shown in the figure: there could be invocations that do not return values which would 
result in effective AWrite invocations by PiQi- One more observation about the situation of 
Figure [5] concerns planarity: though the low-level w and r instances shown may be selected 
subsequences induced by ARead and AWrite operations, the graph of the figure is planar, by 
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arguments similar to those given in the proof of Lemma [4. 1[ Below, this planarity is implicitly 
used in arguments about the transitivity of precedence. 

A next step in the proof is a deduction about AWrite and ARead invocations at ^492, 
many of which are concurrent with the scenario of Figure O a similar deduction can establish 
concurrency with AWrite and ARead invocations at PiQs; more generally, there is a chain of 
deductions about concurrency of operations. To construct this chain of deductions, we depict 
the scenario between piQt and Pje(f_|_i) in Figure El 

P^ef- t W ~ f^^^ 



/ \ / 



/ \ / \ / 

• • • >v</>-(^+i) 



Figure 6: situation for piQt and PiQ{t+i) 

In Figure m processor PiQt^s first ARead, labeled TZ^, is presumed to be a reading of the 
initial registers before Pje{t+i) written them: we suppose this to obtain the worst case 
(fewest number of effective AWrites) for Pie(t+i)'^ behavior. Thus the first ARead at piQt 
influenced by Pie(t+i) '^2, and the dashed arrow from to TZ^ indicates that ^ TZ'^; 
also TZ"^ :< is represented by a dashed arrow, since TZ^ gets the value written by (and 
not by W^, because it cannot be that -< TZ'^). The first AWrite need not be preceded 
by an ARead at PjQ(f_|_i), because the initial state of E is arbitrary. The AReads of processor 
PiQ[t+i) are denoted as 7^-vertices. 

Observation 5.1 Containment properties accompanying the definitions of ~< and ^ relations 
enable the following assertion: an interval from some point in TZ^ through some point in TZ'^~^ 
contains the interval beginning from the end of through the start of W^~^^'^^\ which 
contains the interval o/pj_(f+i) from it through it 

Let It denote the interval from IZ? through TZ^^^. Interval It+i thus goes Observation l5.ll 
can be restated as: interval It contains It+i- By transitivity and a simple induction, interval 
Ii contains It for 2 < t < 0/2 (each step of the induction decreases the number of terms by 2). 
Therefore, if > 2n, we deduce that Ii contains /„, which is an interval of PiQn = Pi- That 
is the linchpin of the proof's argument: the contradicting scenario implies that pi's reading 
of a contaminated variable depends on pi injecting the contamination, which would have to 
continue around the ring. In particular, for line 5's ARead to return a contaminated value at 
Pi, at least one register read by pi would have to be concurrent with a register write by pi 
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due to the AWrite of statement 12, which is not possible. The assumption of a contaminated 
result at line 5 is thereby contradicted, provided (j) >2n. □ 

Corollary 5.2 In any execution of the protocol of Figure\^with (j) > 2n, every invocation of 
ARead((j)) has quasi-atomic behavior. 

Proof: Corollary 14.21 establishes the conditions for quasi-atomic behavior: if piQi invokes 
AWrite at most {(p — 1) times between each of pj's ARead(</>) operations, then pj's AReads 
are quasi-atomic. Arguments given in Lemma IS.lf s proof show, by contradiction, that PiQi 
cannot have (j) > 2n effective AWrite operations concurrent with an ARead((^) by pi. Any 
AWrite operations not concurrent with pj's ARead((/)) have no effect on quasi-atomicity, as 
was explained in the proof of Lemma 14.11 □ 

Lemma 5.3 In any execution of the protocol of Figure with (p > 2n, the number of con- 
secutive _L responses for any pi at line 5 is bounded. 

Proof: We first show, by contradiction, that no execution can have all ARead operations 
return _L: if all ARead ((/>) operations return _L, then eventually the value of x in Figure H] 
remains constant for each pi, throughout the execution. Thus no AWrite operation is effective, 
and no registers are written throughout the execution. Thereafter, every ARead (0) encounters 
no concurrent AWrite; but this implies all low-level reads by any pi obtain the same value, 
which contradicts the assumed return of _L shown in Figure [3l 

Now, again by contradiction, we show that no particular pj's ARead operations continually 
return _L. If pi forever returns _L, then eventually Pi(^i has no effective AWrite operations; 
by induction going around the ring, it follows that PiQi eventually has no effective AWrite 
operations. This contradicts conditions of returning _L in Figure [3l □ 

Theorem 5.4 If (p > 2n and K > 2n, then the two-register adaptation of Dij{(j), K) given in 
Figure is self-stabilizing to mutual exclusion. 

Proof: Having shown that ARead ((/>) has quasi-atomic behavior and the absence of deadlock 
(e.g., no Pi continually encounters _L values for ARead operations), the standard convergence 
arguments for Dij apply: K > 2n implies that eventually pq obtains a value x that exists 
nowhere else in the ring, and this is enough to enforce convergence to mutual exclusion. □ 

6 0(lg n)-Register Adaptation of Dij 

When processor communication using registers and execution is asynchronous, the number 
of reads by pi from p-iQis output registers per effective write is unbounded: pi could be 
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unboundedly faster than piQi, hence many reads get no new mformation. Such scenarios are 
unavoidable, however the Dij protocol of Section [5] uses many reads per effective write even 
in the best case, because ARead{<j)) scans input registers at least 2n times. The point of this 
section is to introduce another Dij adaptation scans input registers O(lgn) times in the best 
case. This can be achieved using ARead(2) and O(lgn) registers between each pair {pi,Pi^i) 
of processors. The basis of the construction is an idea introduced in [T3], which uses a gray 
code [1] representation of the token. The improvement here is a protocol that is simpler to 
reason about than the algorithm of |14] . which instead introduces a parity bit manipulated 
in each write operation, and lacks the formal structure that Lemma 14.31 provides. 

Figure [7] presents the protocol. Each processor pi writes to an array of registers, managed 
by the AWrite/ARead construction of Section 14. li The constant k specifies the number of 
register pairs (Ra[^], RfcH), for < i < k. The register pair for index i corresponds to the 
bit in the gray code representation of a token value. For arguments about the protocol, let 
f^a/hW denote the register pair for bit i. 

The invocation AReadj(2) specifies an ARead(2) invocation on input pair of registers for 
bit i; AWritei(ua/) similarly specifies the output register pair to use for writing. Function 
gray^^ used on lines 7 and 10 decodes the /c-bit gray code representation of a non-negative 
integer; for line 10, gray^^ may encounter a _L value for one or more bits. The convention for 
such cases is that gray^^ maps to _L if AReadj(2) returns _L for any i. 

Three iterations process registers, seen on lines 5, 8, and 18. Whereas the iterations of 
lines 5 and 8 go from to /c — 1, the iteration of line 18 goes in the reverse order: this is 
intentional, and simplifies reasoning about the atomicity of token transfer in a proof. 

The validation of the protocol builds on some simple prop- 
erties and on the definition of a certain type of state in an 
execution. Recall that gray code, like binary arithmetic, orders 
the bits of its representation in order from most significant to 
least significant. Figure [8] shows a 3-bit reflected gray code, for 
example. 

For the local variables defined on line 3 of Figure [3 and for 
the register pair Ra/;,[i], the most significant bit (MSB) has the 
least index. Thus R[A; — 1] represents the least significant bit 
(LSB). Like standard binary encoding, in a sequence of incre- 
ments of a gray code value, the LSB alternates more frequently 

Figure 8: 3-bit gray code 

than does the MSB: 2—1 increments to a k-hit gray code 

changes the LSB 2^~^ times (repeating the sequence of two O's, followed by two I's), whereas 
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000 
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001 
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010 
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110 
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111 
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101 
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100 
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1 Dij,(i^): 



2 constant k = \lgK~\ 

3 local variables X[k], Y[k], x, y 

4 do forever 

5 for i G 0..(A; - 1) 

6 read from output Ra[i] into X[i] 

7 x ^ gray^^(X) mod 

8 for i € 0..(A; - 1) 

9 Y[{\ ^ AReadi(2) 

10 y ^ gray^^(y) mod ivT 

11 if y A i 7^ A x ^ y then 

12 X ^ y 

13 critical section 

14 else if y ^1. Ai = A y = x then 

15 X (x + 1) mod K 

16 critical section 

17 X^grayfc(x) 

18 for i€{k- 1)..0 

19 AWritei(X[i]) 



Figure 7: two register Dij((/>, i^T) protocol for processor pi 

the MSB changes only twice. A useful property of the gray code is that each increment 
changes exactly one bit in the encoding (including rollover from the largest representable 
integer) . 

We define a flash state to be one where all values for the MSB, in any register or any 
internal variable of any processor, are zero. A flash event is the transition from a flash state 
to a non-flash state. A flash event only occurs by the step x ^ (x + 1) mod K in line 15 
of the protocol. After a flash event, po writes the unique one-valued MSB in the ring. A 
home state is one where all values for all bits and corresponding internal variables are equal 
in corresponding bit positions (different bits may have different values, however a bit at any 
position has the same value everywhere). A legitimate state for the protocol is either a home 
state or a successor of a legitimate state. 

Some elementary properties of executions originating from a home state are (z) a home 
state is reached infinitely often, and (ii) all effective AWrite operations are atomic. Properties 
{i)-{ii) can be shown by induction, paralleling standard arguments for the Dij protocol. 
Thanks to property (i) and the definition of a legitimate state, validation of the protocol in 
Figure [7] consists of showing that any execution eventually reaches a home state. Property 



17 



(ii) is technical statement about the conditions of write and effective AWrite operations: at 
most one processor can be engaged in an effective AWrite at any time in an execution of 
legitimate states, and following the completion of an AWrite by pi, processor pi^i correctly 
reads the value before the next effective AWrite. The gray coding ensures that only one AWrite 
can be effective in the iteration of lines 18-19 of the protocol. 

In a legitimate state, a register pair Ra/bi"^] equal except during an AWrite operation, 
which may have written but not yet R^. With respect to any state in an execution, 
a register pair is said to be coherent if both registers have the same value or an AWrite 
operation is underway. Observe that the procedure defining AWrite in Figure [3] ensures that 
both registers are equal upon completion, whether or not the AWrite is effective. Thus, in 
any execution, after each processor has performed all the steps in lines 18-19 of the protocol, 
it follows that all register pairs are coherent for all subsequent states. 

Lemma 6.1 Any execution starting from a flash state contains a home state. 

Proof: We focus on po's behavior for the proof. Only po is capable of changing its most 
significant bit from zero to one, by the assignment of line 15. All other processors copy input 
register values to output register values. The proof of the lemma is in two parts: first, we 
show that po eventually does change the MSB, that is, that a flash event occurs; the second 
part is to show that a home state is reached sometime after the flash event. 

The inevitability of a flash event is shown by contradiction. Suppose po never changes its 
most significant bit. After some writes of other bits, po has no effective writes throughout 
some suffix of the execution, because line 15 does not execute infinitely often by assumption. 
It follows that eventually there is a suffix where pis output registers have the same values 
as po's output registers, as pi will copy these values in some cycle of the protocol (line 12) 
— there cannot be a _L-value read when there is no concurrent write by pq. By induction, pi 
for < i < n eventually also has the same output registers as po, and no processor will have 
any effective write for the remainder of the execution. However, such a condition contradicts 
the condition of line 14 for processor po, implying that a flash event must occur. 

A flash event has po assigning one to the MSB, thus writing Ra[0] ^ 1 and Rfc[0] ^ 1. 
After the AWrite operation at pQ associated with this flash event, the MSB of po is the only 
MSB with 1. In fact, po will not again perform an effective write until this 1 value propagates 
through the ring (for instance, Pn-i has for the MSB, and does not engage in an effective 
write until it copies 1 from Pn-2)- Consider the event of pi reading the 1 MSB from pQ by an 
ARead(2) operation. This ARead has quasi-atomic behavior because the two low- level writes 
to Ra and R;, of a single AWrite by po cannot be concurrent with all four low-level reads of 
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the ARead operation. Furthermore, all the AWrite operations to less significant bit positions 
occur before the AWrite of the MSB, which implies that after pi reads 1 for the MSB, all 
the other bits that pi reads are atomic and have the values written by pq. Inductively, this 
argument holds for the transfer of values from pi to pj©i, up to Pn-i- Finally, after Pn-i 
writes 1 for its MSB, we infer that all values at all positions are the same throughout the 
ring, which is a home state. □ 

Lemma 6.2 Any execution contains a flash state. 

Proof: Using arguments (based on contradiction) similar to those in the proof of the pre- 
vious lemma, po executes line 15 infinitely often in any execution, so the MSB of pq changes 
throughout the execution. To show that a flash state occurs, we consider pQ invoking an 
effective AWrite(O) and deduce that pi copies its MSB from pQ, then p2 copies its MSB from 
pi, and generally PiQ)i copies from pi, all before po invokes AWrite(l); this shows that a flash 
state is reached, provided the copying of MSBs occurs in sequence, so that all are zero valued. 

After each token increment (line 15), po writes the token value to output registers and 
waits until the same value is read from Pn-i- A property of the gray code is that the LSB 
changes in half of the token increments. Since 2^ > K, the LSB changes at least K/2 > n 
times between consecutive AWrite(O) and AWrite(l) operations of the MSB. Put another way, 
Po expects to observe at least n changes of the LSB in this period. The question is, which of 
these changes are due to contaminated reads {e.g., an ARead at po concurrent with multiple 
AWrite operations by pn-i), which are due to LSB values initially present in processors other 
than Pq, and which are values propagated around the ring, from po back to pq. By counting 
these types of changes, we shall bound the number of values not propagated around the ring, 
showing them to be at most n in total. 

Suppose Pq does not write any registers after the AWrite(O) of the MSB completes; we 
count the number of LSB changes that po could observe during the subsequent execution. 
The count is derived inductively, starting with the number of LSB values observed by pi. 
The case for pi is simple because we suppose po writes once. Processor pi may observe an 
initial value, and then another value that po writes. We ignore the case of reading _L, because 
the protocol of Figure [71 Since the LSB is written at most once by po, each ARead(2) by pi is 
atomic, so no contaminated reading occurs. The conclusion is that pi observes at most two 
values for the LSB. Each such observed value at pi could result in an effective AWrite of its 
LSB. 

Counting the observable values for p2 introduces contaminated values: because pi may 
write the LSB register pair twice, p2 could read a contaminated value, however. Lemma 14.31 
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limits to one the number of contaminated reads. If p2 does read a contaminated value, it 
follows that the correct value would be observed by another ARead(2). Another scenario for 
P2 is the absence of contaminated values, in which case p2 may observe both values written 
by pi. The total number of observable values is three in either scenario: one for the initial 
value, followed by two more observed values due to pis writes. 

The induction hypothesis is that pi may observe at most i + \ values in the execution 
where pq does not write any registers. Assume that pi^i observes and writes at most i values 
for the LSB. As pi reads the values it is possible that some (or all) of the writes are concurrent 
with piS ARead(2) operations, resulting in contaminated reads. Again, Lemma [4.31 limits the 
number of contaminated values to be at most half the number of AWrite operators by Pi-i. 
It follows that in any scenario, pi observes at most i values due to Pi-is writes. The total 
number is i + 1 because pi can also observe the initial value of the MSB. 

The conclusion from the induction is that pQ "observes" at most n changes to the LSB 
read from Pn-i (these would not be actually observed because we suppose po does not write 
any registers). Note that if all n changes due to initial values and operations by pi-pn-i 
without influence of pq are observed first at before any influence of values written by po 
circulate the ring, then the MSB at po retains the value 0, because more than n changes 
of the LSB are needed to enable AWrite(l) of the MSB. It remains to consider more rapid 
influence of values written by pq affecting what other processors write. Any values copied 
directly or indirectly from pi to Pi+i do so only for non _L-ARead(2) operations; and since pq 
writes the MSB once in the execution under examination, it follows that any such copying 
obtains the value for the MSB. Therefore, after n changes to the LSB by the next 

change of the LSB is due to a value circulating the ring, from po to Pn-i- Each ARead(2) 
operation influenced by po values includes an atomic reading of the MSB copied from po, 
hence the (n + l)*'^ change to the LSB is accompanied, if not preceded, by Pn-i writing to 
its MSB output pair. This establishes a flash state. □ 

Theorem 6.3 If K > 2n, then the 0(}gK) -register adaptation of Dij(K) given in Figure^ 
is self-stabilizing to mutual exclusion. 

Proof: Every execution of the protocol has a suffix in which all states have coherent regis- 
ters. Within such a suffix, Lemma [6.2l is applicable, guaranteeing that a flash state eventually 
occurs. Subsequently, Lemma l6.ll asserts that a home state will be reached, whereafter reg- 
isters behave atomically, because at each state the choice of what register pair will next be 
effectively written is deterministic, and once the AWrites of lines 18-19 complete, the result 
will be atomically read before the next effective write is enabled. Thus, standard arguments 
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for Dij apply to show safety. 



□ 



7 Discussion 

The protocols of Section [4. II use well known techniques for register constructions: duplicating 
written values and multiple scans by the reader are standard fare in the literature. The 
adaptation in Section [5] takes advantage of inherent limitations on register writing, even for 
an illegitimate state, of the Dij protocol. Section [6] exploits two more standard techniques 
from the literature of register constructions, representing a value with bit registers (where 
safe and regular properties coincide) and the idea of ordering writes and reading scans in 
opposite directions [H [10] . 

Ideas for limiting concurrency, particularly in common shared memory models, include 
counting or balancing networks and filters in mutual exclusion algorithms. However the 
technique use here is different, being geared to the Dij protocol. One might therefore consider 
the protocols of this paper to be of very limited use in other contexts. However, the history 
of self-stabilization literature should be consulted before such a judgment. Generalizations of 
the token ring lead to wave protocols (propagation of information with feedback), and other 
synchronization or control algorithms. Several of the crucial properties of Dij are enjoyed 
by other self-stabilizing (and non-stabilizing) protocols, including implicit restrictions on 
concurrency. For instance, for many protocols, quiescence of selected processes results in 
deadlock, so there is hope that counter-flushing |12j or similar techniques could simplify the 
adaptation to safe-register communication. 

There have been relatively few investigations of wait-free self-stabilization or stabilization 
in the common shared memory model: papers appear sporadically over the years since Dij 
first appeared [8l [TTl [15]. This intersection of topics appears to contain many unresolved 
questions. 
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